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This report summarizes the results of a survey of 
school supervisors who evaluated teachers in their schools who were 
graduates of one of Michigan State University's (MSU) teacher 
preparation programs. The major sections of the survey questionnaire 
provided: (1) background information (e.g.. How many times have you 
observed this MSU graduatv teach?); (2) teacher's knowledge (e.g., 
How would you rate this teacher's knowledge of the subjects he or she 
teaches?); (3) teacher's competence (e.g.. How would you rate this 
teacher 1 s competence in designing lessons?); *(4) teacher's 
professional actions (e.g. , How often does this teacher initiate 
discussions with colleagues that center on teaching?); (5) teacher's 
school environment (e.g. f Dees the teacher's school environment 
encourage planning lessons that stress student creativity?) ; and (6) 
open ended questions (e.g.. Are there areas of teaching that should 
have been more adequately addressed in the preparation program?) . 
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SUPERVISORS OF TEACHER EDUCATION GRADUATES 
EVALUATE MSU'S PREPARATION PROGRAMS 



This study is an extension of a comprehensive program evaluation effort at 
Michigan State University (MSU). The overall evaluation design (see Freeman, 
1986) traces the progress of students from the time they enter one of MSU f s 
five teacher preparation programs (Standard Program, Academic Learning, 
Heterogeneous Classrooms, Learning Community or Multiple Perspectives Program) 
through five or six years following graduation. In Spring, 1987, the Office of 
Program Evaluation conducted a follow-up study of 1984-1986 teacher education 
graduates (see West, 1987) to obtain pertinent information such as employment 
histories and preparation program critiques. As part of that follow-up study, 
respondents were asked to provide the name and address of their current 
supervisor. This report summarizes the results of a subsequent survey of 
these administrators, findings that should contribute to the ongoing 
development and improvement of Michigan State's preparation programs. 

PROCEDURE 

Instrument: The "1987 Follow-Up Study of M.S.U. Graduates - Supervisor's 
Survey" is a 72 item questionnaire designed by Don Freeman and the 
Undergraduate Program Evaluation Committee (UPEC) . Major sections of the 
questionnaire provide: 

(a) Background Information (e.g., How many times have you observed this 
MSU graduate teach?) 

(b) Teacher's Knowledge (e.g., How would you rate this teacher's knowledge 
of the subjects s/he teaches?) 

(c) Teacher's Competence (e.g., How would you rate this teacher's 
competence in designing lessons?) 

(d) Teacher's Professional Actions (e.g., How often does this teacher 
initiate discussions with colleagues that center on teaching?) 

(e) Teacher's School Environment (e.g., Does the teacher's school 
environment encourage planning lessons that stress student 
creativity?) 

(f) Open Ended Questions (e.g., Are there areas of teaching that should 
have been more adequately addressed in the preparation program?) 



Sample: In the original follow-up study 540 graduates were in the sample 
and 254 returned usable responses. Of this group of 254 MSU Alumni, 116 are 
currently full time teachers, 84 of whom (73%) provided the name and address of 
their current supervisor. Thus, the sample for the supervisor survey was 84 
supervisors of graduates who are currently teaching full-time. The 
questionnaire was sent in April, 1987 and a second mailing was conducted four 
weeks later to all supervisors who had not replied by that time. Ultimately, 
62 supervisors (74% return rate) returned the questionnaire. 

Because some supervisors chose to skip questions, the usable sample was 
generally smaller for any given analysis than may be implied from the 62 usable 
returns. Also, one must keep in mind that approximately 25% (22 out of 84) 
chose not to participate. And of the 116 full time teacher- graduates , 32 did 
not list the name/address of their supervisor. It is undetermined whether 
these nonpar ticipants were systematically different from the supervisor 
respondents. 

Statistic! Analyses: The primary purpose of the statistical analyses was 
to summarize supervisors 1 responses to the survey. Therefore, most of the 
summary information presented in this report is based on frequency counts 
tabulated for each response on each question. However, in many cases, mean 
scores and t- tests were also determined to compare two groups of supervisors; 
namely those who supervise graduates of the (1) Standard Program or (2) 
Alternative Programs. Because the Alternative Programs have many common 
features and a relatively small number of graduates compared to the Standard 
Program, supervisors of graduates from the four Alternative Programs were 
re-grouped as "Alternative Programs" in these comparisons. The probability of 
a Type I error was fixed at .05 as a basis for deciding which contrasts would 
be presented in this report. 

The statistical analyses cited in this report are not meant to exhaust all 
possibilities of important findings. Rather, this report is intended to 
provide an overview of some of the results that appear most significant at this 
time . 

BACKGROUND INFORMATION 
The 62 supervisors reported that 58% of the graduates they were asked to 
review joined their respective staffs during the academic year of the survey, 
39% were in their second year of teaching, and 3% joined the staff three years 
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earlier. Most supervisors (85%) were aware that the teachers were MSU 
graduates and 65% said they were also aware of the specific teacher education 
program the graduate^ completed. Having had experience supervising at least 
one MSU graduate, supervisors were asked if they would encourage their school 
district to hire other MSU graduates. Forty percent replied definitely yes, 
24% probably yes and 35% replied they did not know enough about the program to 
make this judgement. None of the supervisors indicated that they would decline 
to encourage their district to hire an MSU graduate. 

Because it was important to know the basis on which judgements were made, 
supervisors were asked, "About how many times have you watched this M.S.U. 
graduate teach?" TABLE I indicates the frequency of observations in the last 
two years: 



TABLE I 



Freauencv of Observation of Graduates 




percetit 


Once or Twice 


2% 


Three or Four Times 


18 


Five or Six Times 


26 


Seven to Ten Times 


24 


More than Ten Times 


31 



When supervisors were asked whether this number of observations is consistent 
with the frequency each usually observes beginning teachers, 91% replied 
,f yes." Only 5% indicated that the number of observations was higher than 
usual . 
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GRADUATES' KNOWLEDGE AND COMPETENCE 
One important dimension of program evaluation considers how graduates 
perform certain teaching tasks. Supervisors were asked to rate the performance 
of graduates in both knowledge and competence areas on the following scale: 

1 - Exceptional 

2 - Strong 

3 - Above Average 

4 - Adequate 

5 - Inadequate 

8 - Insufficient Information 



TABLE II presents the supervisors' ratings of GRADUATES ' PROFESSIONAL KNOWLEDGE 
and TABLE III the ratings of GRADUATES' TEACHING PERFORMANCE. 



TABLE II 

Supervisors' Ratings of Graduates' Professional Knowledge 



Knowledge of Subject Taught 


N 
62 


MEAN 
1.98 


S.D, 
.77 


Knowledge of Educational 
Concepts and Theories 


61 


2.36 


.91 


Knowledge of Professional 
Practice/Pedagogy 


61 


2,39 


1.00 


Knowledge of Recent Research 


48 


2.68 


.90 



Rating^: 1-exceptional, 2-strong, 3-above average, 4-adequate, 
5-inadequate. 
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TABLE III 

Supervisors' Ratings of Graduates' Teaching Performance 



COMPETENCE AREA 


WEAN 


S.D. 


Establishing effective working relations with students 
from diverse cultural and academic backgrounds. 


2.11 


1.03 


Designing lessons, units and courses of study. 


2.16 


1.01 


Deciding what content to teach and what not to teach. 


2.18 


.86 


Making other instructional decisions in a sound and 
defensible manner. 


2.37 


1.02 


Communicating with parents. 


2.38 


1.09 


Applying effective methods of teaching specific subjects. 


2.40 


.98 


Teaching in a way that insures that most students have 
a thorough understanding of the content, structure, 
and significance of the subject matters. 


2.46 


.88 


Establishing a classroom environment in which students 
actively take responsibility for themselves and others. 


2.49 


1.16 


Analyzing and improving his/her teaching performance. 


2.50 


.93 


Motivating students to participate in academic tasks. 




1 • I/O 


Applying effective classroom management methods. 


2.50 


l.ll 


Assessing student learning and development. 


2.51 


.88 


Organizing instruction in ways that insure smooth 
transitions between different learning activities. 


2.51 


] ,03 


Planning instruction that addresses the diverse needs 
and achievements of students. 


2.54 


.95 



OVERALL: 2.400 ,999 

Rating?: 1-exceptional, 2-strong, 3-above average, 4-adequate, 5-inadequate. 



As illustrated in TABLE II, the mean scores in the four knowledge areas 
were all "above average" (the lower the mean score, the closer to 
"exceptional"). Supervisors were more consistent in their ratings of 
"Knowledge of Subject Taught" than in their judgements of graduates 1 
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"Knowledge of Professional Practice/Pedagogy." Thirteen supervisors, 
indicated they did not have sufficient information to make a judgement about 
the graduates' "Knowledge of Recent Research in Education." 

As in the ratings of graduates 1 knowledge, supervisors rated all teaching 
performance areas (Table III) between "above average" and "strong." The means 
were remarkably similar across the 14 areas of performance. Said another way, 
there were no particular areas that were rated as far stronger or far veaker 
than the others. 

When the two groups of supervisors (i. . , those who supervise graduates of 
the Standard Program and those who supervise Alternative Program(s) graduates 
were compared, there were no significant differences (alpha-. 05) between the 
groups on any of the knowledge or performance measures. 



GRADUATES 1 PROFESSIONAL ACTIONS 
Teachers sometimes distinguish thems*> n m by doing certain things more 
often than their colleagues. Supervisors were asV J to ju^ge if MSU graduates 
take various actions (1) more often, (2) about the same number of times 
(average), or (3) less often when compared with other teachers in the 
building. TABLE IV summarizes tha supervisors 1 ratings of the professional 
actions of graduates. The actions are ordered from those which exceed the 
average of other teachers in the building to those for which MSU alumni are 
least likely to exceed the norm. 



TABLE IV 

• Supervisory Ratings of Graduates 1 'Professional Actions 



^HONS MEAN S.D. 

Continually strives to upgrade his/her teaching, 1.43 ,56 



Encourages students to accept responsibility for 1.55 
their own beliefs and actions. 



,57 



Presses students for high levels of academic I.57 ,56 
achievement . 

Initiates discussions with colleagues that center I.59 ,53 
on t^coMng. 

Reflects on his/her own teaching practices. 1 # 60 .62 

Interacts with students outside the classroom settings. 1.61 .59 

us- 



Table IV, (Corit.) 

Communicates with parents. 

Plans lessons that develop problem solving 
skills. 

Uses student-centered approaches to instruction. 

Makes rational, data-based decisions in addition 
to relying on intuitions. 

Seeks opportunities to participate in professional 
in-service activities. 

Seeks opportunities to work with youngsters 
from culturally diverse backgrounds. 

Applies the "cooperative learning" model of 
instruction. 

Incorporates school/district policies into 
his/her own teaching practices. 

Plans lessons' that stress student creativity. 

Expresses a concern for equity or other issues 
of social justice. 

Voxunteers to serve on school or district committees. 

Raises questions about why things are being done 
as they are or why certain actions are not 
being taken. 

Assigns homework. 

Designs games or other instructional materials. 

Plans units that address interdisciplinary 
problems or themes. 

Uses community resources as a supplement to 
classroom instruction. 

Reads professional journals. 

Participates in professional association/union 
activities . 

OVERALL: 

Ratings : l-*more often, 2-average, 3-less often 
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As shown in TABLE IV, supervisors rate MSU graduates as engaging in nearly 
all of the listed professional actions more often than other teachers in their 
buildings. However, MSU graduates ranked very close to the mean on six of the 
24 actions that were cited and below the average frequency for one other area, 
"Participates in professional association/union activities 11 . There were no 
significant differences (alpha-. 05) in ratings of graduates of the Standard 
Program and alumni from Alternative Programs. 

. GRADUATES 1 SCHOOL ENVIRONMENT 

Teacher preparation programs are sometimes criticized for their failure to 
recognize the "realities" of the school context. In order to provide a better 
sense of the context in which our graduates work, supervisors were asked to 
describe the extent to which beginning teachers are encouraged by the "school 
context" to pursue 22 professional activities. Supervisors recorded their 
ratings on a four-point scale, where 1-strongly encouraged, 2-encouraged, 
3-neither encouraged nor discouraged, and 4-di scour aged. 

The results summarized in TABLE V are presented in rank order from 
accivities that are most likely to be encouraged to those are lease likely to 
be encouraged. The first column of Table V also cites the ranking of a given 
action on the frequency of occurance scale, where 1-the action MSU graduates 
were most likely to engage in more often than other teachers (see Table IV) . 
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TABLE V 

Supervisory Ratings of School Context 



FREQUENCY 
OF 

ACTIONS 
RANK 
ORDER 
5 



CONTEXT 
RANK 
ORDER 
1 



Ratings : 1 « Strongly Encouraged 

2 - Encouraged 

3 - Neither Encouraged 

nor Discouraged 

4 - Discouraged 



ACTIVITY 

Communicate with parents 



MEAN 
1.23 



S.D . 
.49 



1 


2 


Encourage students to accept responsibility 
for their own beliefs and actions. 


1 


.31 


.50 


2 


3 


Press students for high levels of academic 
achievement . 


1 


.32 


.54 


12 


4 


Incorporate school or district policies into 
teaching practices. 


1 


.34 


.60 


9 


5 


Participate in professional in-service 
activities. 


1 


.37 


.52 


6 


6 


Plan lessons that develop problem solving 
skills. 


1 


.40 


.58 


3 


7 


Reflect on his/her own teaching practices. 


1 


44 


.60 


8 


8 


Make rational, data-based decisions in 
addition to relying on intuitions. 


1 


53 


.65 


13 


9 


Plan lessons that stress student creativity. 


1. 


61 


.66 


7 


10 


Use student-centered approaches to instruction. 


1. 


63 


.66 


15 


11 


Volunteer for school or district committees. 


1. 


66 


.63 


20 


12 


Use community resources. 


1. 


67 


.68 


16 


13 


Raise questions about why things are being 
done the way they are or why certain actions 
are not being taken. 


1. 


72 


.66 


4 


14 


Interact with students outside the classroom 
setting. 


1. 


81 


.79 
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11 15 Apply the "cooperative learning" model of 1.85 .71 

instruction. 

17 16 Assign homework. 1.88 gg 

18 17 Design games or other instructional ^materials. 1.93 .70 

14 18 Express a concern for equity or other issues 1.98 .77 

of social justice. 

21 19 Read professional journals. 2.07 .72 

10 20 Seek opportunities to interact with youngsters 2.08 .85 

from culturally diverse backgrounds. 

19 21 Plan units that stress interdisciplinary 2.17 .78 

problems or themes. 

22 22 Participate in professional association/union 2.26 .80 

activities . 

OVERALL: 1-693 ~ 

ir : : : 

Two actions, "Continually strives to upgrade his/her teaching" and "Initiates 
discussions with colleagues that center on teaching" were not included in the social 
context subscale and hence not included in TABLE V. 



As shown in TABLE V, supervisors reported that none of the activities cited 
on the survey are discouraged in the school districts in vhich graduates 
teach. All activities are encouraged, albeit some more than others. 

Supervisors said, with little variance, that the schools in which graduates 
teach are most likely to strongly encourage "communicating with parents," 
"encouraging students to accept responsibility for their own beliefs and 
.actions," and "pressing students for high levels of academic achievement." The 
school context is least likely to encourage "read professional journals," "seek 
opportuniites to interact with youngsters from culturally diverse backgrounds," 
"plan units that stress interdisciplinary problems or themes," and "participate 
in professional associations /union activities." 

When the supervisors ' rank order of the extent to which the school context 
encourages a particular activity is compared to the rank order of supervisors' 
judgements of the frequency of graduates' activities in these same areas, there 
is general agreement. That is, the frequency of graduates' actions was 
typically in harmony with the press to engage in that activity within the 
school context. 
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But there were at least some "outliers" or anomalies. For example, 
"interacting with students outside the classroom" is ranked 14th out of 22 
items on the school context scale, but ranks fourth on the frequency of 
occurance scale. Similarly, "Seeking opportunities to interact with youngsters 
from culturally diverse backgrounds" ranked 20th on the contextual environment 
scale and 10th on the frequency of occurance scale. The more troublesome 
contrasts are those in which the contextual press to do something ranks higher 
than the frequency of occurance of that activity. See. for example, tae 
comparative ranks for "using community resources" and "communicating with 
parents . " 

PROGRAM CRITIQUE 

The final section of the questionnaire consisted of two open-ended 
questions: 

(1) Has this MSU graduate demonstrated "excellence" in any area(s) of 
teaching? , and 

(2) Are there areas of this teacher's performance that you feel should 
have been addressed more adequately in the preparation program? 

Forty five supervisors (73%) replied to one or both of the above questions. 
Each cited one or more areas of "excellence" and/or areas that the preparation 
program(s) could have addressed more adequately. 

Of 59 comments indicating one or more areas of "excellence", 12 statements 
commented on the high quality of interpersonal relationships with 
students/parents/peers (e.g ; , working with parents, positive staff relations, 
etc.); three statements mentioned graduates 1 actions in volunteering for school 
committees or other extra-duties, and 12 comments noted personal qualities of 
graduates such as confidence, leadership or self- assessment skills. Most 
statements of "excellence," however, (32) noted graduates' knowledge in 
specific subjects areas, such as science, math, physical education, and art or 
"knowledge of subject matter in general." 
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Twenty seven supervisors wrote 33 statements about areas each felt could 
have been addressed more adequately in the graduates* preparation program. 
Areas addressed by the 33 statements included: 

Methods/techniques of teaching (e.g., lack of knowledge about motivation, 
I.T.X.P., child behavior, guidance skills, creativity, student 
assessment) (12) 

Management/control/discipline/classroom organization (10) 

Personal qualities (e.g., lack of confidence, very nervous, no leadership 
skills) (5) 

Interest in applying research findings (3) 

Professionalism is the classroom (1) 

Knowledge of certification requirements (1) 

Staff communication (1) 



CONCLUSIONS 

The ratings by supervisors of graduates' (1) professional knowledge and (2) 
teaching performance were remarkably similar. Mean ratings for survey items in 
both areas were consistently above average. 

With little variance, supervisors' rated graduates' knowledge of the 
subjects they teach as "strong" (x - 1.98) but tended to rate graduates' 
knowledge of recent research in education somewhat lower (x - 2.68). Among the 
areas of teaching performance cited on the survey, supervisors were especially 
impressed with our graduates' competence in (a) establishing effective working 
relationships with students from diverse cultural and academic backgrounds, (b) 
designing lessons, units and courses of study, and (c) deciding what content to 
teach and what not to teach. 

Supervisors also reported that MSU graduates, as a group, take various 
professional actions more often than the average teacher in their respective 
buildings. This was especially true of their efforts to: ^a) upgrade their 
teaching, (b) encourage students to accept responsibility for their own beliefs 
and actions, (c) press students for high levels of academic achievement and (d) 
initiate discussions with colleagues that focus on teaching. 
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The frequency of various professional "actions" typically paralleled the 
extent to which these actions were encouraged within each school context. 
However, there were some exceptions. Whereas supervisors said the school 
context was most likely to strongly encourage "communicating with parents," the 
highest priority for the beginning MSU graduate teacher was "striving to 
upgrade teaching." 

Seventy-three percent (73%) of the supervisors wrote comments about the 
relative strengths and inadequacies of MSU preparation programs as reflected in 
actions of graduates. Supervisors appeared to be impressed with the subject 
matter knowledge graduates demonstrate:, but felt certain areas of the 
graduates 1 preparation could have been improved, such as: classroom 
organization and management skills and specific methods of teaching. 
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